Field of the Invention
The present invention relates to a method of reducing an amount of learning data which are required for use in a learning procedure applied to a neural network which is to perform pattern recognition.
When a neural network is to be used in pattern recognition applications, it is necessary to execute beforehand a learning procedure, for establishing suitable parameter values within the neural network. In the learning procedure, a set of sample patterns (referred to herein as the learning data), which have been selected in accordance with the patterns which are to be recognized, are successively inputted to the neural network. For each sample pattern there is a known appropriate output pattern, i.e. which should be produced from the network in response to that input pattern. The required known output patterns will be referred to as the teaching data. In the learning procedure, the learning data patterns are successively supplied to the neural network, and resultant output patterns produced from the neural network are compared with the corresponding teaching data patterns, to obtain respective amounts of recognition error. The internal parameters of the neural network are successively adjusted in accordance with these sequentially obtained amounts of error, using a suitable learning algorithm. These operations are repetitively executed for the set of learning data, until a predetermined degree of convergence towards a maximum degree of pattern recognition is achieved (i.e. the maximum that can be achieved by using that particular set of learning data). The degree of recognition can be measured as a recognition index, expressed as a percentage.
The greater the number of sample patterns constituting the learning data, the greater will be the invariant characteristic information that is learned by the neural network. Alternatively stated, a learning algorithm which is utilized in such a procedure (i.e. for adjusting the neural network internal parameters in accordance with the error amounts obtained during the learning procedure) attempts to achieve learning of a complete set of probability distributions of a statistical population, i.e. a statistical population which consists of arbitrary data, consisting of all of the possible patterns which the neural network will be required to recognize after learning has been achieved. That is to say, the learning algorithm performs a kind of pre-processing, prior to actual pattern recognition operation being started, whereby characteristics of the patterns that are to be recognized are extracted and applied to modify the internal parameters of the neural network.
In the prior art it has been necessary to utilize as large a number of sample data in the learning procedure as possible, in order to maximize the recognition index that is achieved for the neural network. However there are practical limitations on the number of sample patterns that can be stored in memory for use as learning data. Furthermore, such learning data may include data which will actually tend to lower the recognition index, if used in the learning procedure.